On the Interaction Between True Source, Training, and Testing Language Models

An interaction has been found between the true source language model, training language model, and the testing language model. This interaction has implications for vocabulary independent modeling, testing methodologies, discriminative training, and the adequacy of our current databases for continuous speech recognition (CSR) development. The current DARPA databases suffer from the described difficulties which suggests that new CSR databases are needed if we are to further advance the state-of-the-art. The Interaction During Training When a category model (e.g. a context-free (CF) model such as a monophone) is used to a model a set of subcategories (e.g. context-dependent (CD) models such as triphones), the category model becomes the subcategory prior-probability weighted average of the subcategory models: Meat E PsubeatMsubcat where M denotes a model. (The mathematics used here are intended to be conceptual rather than rigorous. Thus models will be considered to be averages. In practice, the method for deriving a model from a set of sub-models or observations is highly dependent upon the form of model used.) In a field, such as speech recognition, where models are trained from exemplars, the subcategory model will generally be: N 1 Msttbcat = ~ ~.= Osubeat,i where 08=bcat,i is an observation emitted from the subcategory. Mcat combines both the subcategory models and the prior-probability of the subcategories and similarly Msubcat combines the observations and their (sampled) prior-probabilities. *This work was sponsored by the Defense Advanced Research Projects Agency. In speech recognition, a phone category would contain some set of subcategories and a subcategory would defined by some specific set of context factors. There are many factors which may be used to define the subcategories [3]; a commonly used set is triphone [18] subcategories and monophone categories. Alternatively, stressed and unstressed phones might be combined. (Note that this averaging is recursive: subcategories are the combination of some set of subsubcategories and so on...) We assume that speech is generated from some "true source" language model. (This language model would change as a function of many factors such as topic, history, and participants, but we will assume it to be constunt for each task.) This true language model is known for some artificial tasks such as the DARPA Resource Management (RM) database [16], but can be estimated for naturally elicited speech and text if sufficient data is available. (However, current techniques for estimating language models are fairly rudimentary.) Since the acoustic realization of the phones will be a function of this true language model, any acoustic models averaged over any group of subcategories will learn this true language model to some degree. (Learning the language model "to some degree" may be viewed as favoring the more likely subcategories.) Pragmatically, we have insufficient data to model all relevant subcategories separately and, even if we had sufficient data, we currently have insufficient computational resources to process all of it in any practical manner. Thus, since we must combine subcategories into larger models, a recognition system would favor the subcategories that were more commonly observed during training. Implications for Performance Testing Recognition is performed using some explicit language model. (No-grammar is a language model in which all following words are equally likely.) If the performance of a system is tested using a weaker language model than the true source language model, the acoustic models, if they have been affected by the training data language model, will strengthen the the total language model in the recognizer. Thus, one would expect better recogni-